Robotic system for flexible endoscopy

ABSTRACT

A robotic manipulator  100 , controller  300  and system for use in flexible endoscopy, the manipulator  100  comprising a flexible member configured to be coupled to an endoscope, and an arm connected to and movable by the flexible member, wherein the flexible member has a first end connected to the arm and a second end connectable to the controller  300  to allow a physical movement of the arm to be controllable by a physical movement of the controller  300.

FIELD OF THE INVENTION

The present invention relates to a robotic system for flexible endoscopyand in particular but not exclusively to robotic manipulators,controllers, systems, methods and uses thereof for performing surgery.

BACKGROUND OF THE INVENTION

In line with minimally invasive surgery (MIS), flexible endoscopy isused to inspect and treat disorders of the gastrointestinal (GI) tractwithout the need for creating an artificial opening on the patient'sbody. The endoscope is introduced via the mouth or anus into the upperor lower GI tracts respectively. A miniature camera at the distal endcaptures images of the GI wall that help the clinician in his/herdiagnosis of the GI diseases. Simple surgical procedures (likepolypectomy and biopsy) can be performed by introducing a flexible toolvia a working channel to reach the site of interest at the distal end.The types of procedures that can be performed in this manner are limitedby the lack of manoeuvrability of the tool. More technically demandingsurgical procedures like hemostasis for arterial bleeding, suturing tomend a perforation, fundoplication for gastrooesophageal reflux cannotbe effectively achieved with flexible endoscopy. These procedures areoften presently being performed under open or laparoscopic surgeries.

Endoscopic submucosal dissection (ESD) is mostly performed usingstandard endoscope with endoscopically deployed knifes. Performing ESDthus requires tremendous amounts of skill on the part of the endoscopistand takes much time to complete. Furthermore, constraint in instrumentalcontrol makes it prone to procedural complications such as delayedbleeding, significant bleeding, perforation, and surgical proceduralcomplications. Although ESD is increasingly recognized as an effectiveprocedure for the treatment of early-stage gastric cancers, due to theseproblems, ESD remains a procedure performed only by the most skilledendoscopists or surgeons. Severe limitations in the manoeuvring ofmultiple instruments within the gastric lumen pose a major challenge tothe endoluminal operation. Natural force transmission from the operatoris also hampered by the sheer length of the endoscope, resulting indiminished, and often, insufficient force at the effector end foreffectual manipulations. Besides, as all instruments are deployed inline with the axis of the endoscope, off-axis motions (e.g.triangulation of the instruments) are rendered impossible.

Natural Orifices Transluminal Endoscopic Surgery (NOTES), a surgeryusing the mouth, anus, vaginal, nose to gain entry into the body) is amethod used for surgery that does not require any percutaneous incisionson the abdominal wall. However, for NOTES to be used on human safely,many technical issues need to be addressed. Out of which tooling forfast and safe access and closure of abdominal cavity and spatialorientation during operation are of paramount importance.

With the invention of medical robots like the Da Vinci surgical systems,clinicians are now able to manoeuvre surgical tools accurately andeasily within the human body. Operating from a master console, theclinician is able to control the movements of laparoscopic surgicaltools real time. These tools are commonly known as slaves. However,master-slave surgical robotic systems are rigid and the slavemanipulators enter the human body by means of incisions.

Diseases of the GI tract such as, for example, peptic ulcer, gastriccancer, colorectal neoplasms, and so forth, are common in mostcountries. These conditions can be diagnosed with the aid of theflexible endoscopes. Endoscopes incorporate advanced video, computer,material, and engineering technologies. However, endoscopists oftenstill complain of the technical difficulties involved in introducinglong, flexible shafts into the patient's anus or mouth and there isstill a tool lacking to carry out GI surgeries without creating anincision in the human body and over as short a time as possible sincetime is an essence during acute GI bleeding.

SUMMARY OF THE INVENTION

The present invention is defined in the appended independent claims.Some optional features of the present invention are defined in theappended dependent claims.

According to a first aspect, the present invention relates to a roboticmanipulator for flexible endoscopy comprising:

-   -   a flexible member configured to be coupled to an endoscope, and    -   an arm connected to and movable by the flexible member,        wherein the flexible member has a first end connected to the arm        and a second end connectable to a controller to allow a physical        movement of the arm to be controllable by a physical movement of        the controller.

The robotic manipulator may be used to perform intricate and precisesurgical intervention. It may be inserted into the tool channel ofexisting endoscopes or attached in tandem to the endoscope. The roboticmanipulator may allow for real time endoscopic view, thus providing theadvantage to the endoscopist for performing more intricate and difficultsurgical procedures using natural orifices to access the internalorgans. In particular, the GI tract thus eliminating any scars on thepatient.

In various embodiments, the robotic manipulators may be attached to theendoscope and introduced into the patient together. In other exemplaryembodiments, the endoscope may be introduced first into the patient withthe manipulators being introduced after the site of interest has beenreached. In various exemplary embodiments, hollow, flexible tubes can beattached to the endoscope, and manipulators can be threaded throughthese tubes to reach the distal end. The endoscope also provides toolchannels for instruments to go through which enable the endoscopist toperform a variety of treatments such as biopsy, polypectomy, marking,haemostasis, etc. Some endoscopes have two tool channels that canpotentially accommodate two robotic arms. Alternatively, endoscopes maybe custom designed to accommodate the robotic manipulator. Accordingly,the term “coupled to” in relation to the interrelationship between theflexible member and the endoscope covers fixed attachments, removableattachments, or even simple contacting or supporting dispositions.

The endoscope may be used to inspect, diagnose and treat variouspathologies in the upper or lower GI tract. A typical endoscope includesan ultra compact Charged-Coupled Device (CCD) camera, light source and achannel for infusing or withdrawing liquid or gas from the patient'sbody. The tip of the endoscope may also be steerable so that theendoscope may be able to transverse through the winding channel of theGI tract faster, safer and giving less pain to the patient.

The provision of the flexible member that forms part of the roboticmanipulator may enable the robotic manipulator to be introduced intandem with a flexible endoscope via natural orifices (e.g. mouth andanus) to access the GI tract to perform dexterous procedures allowingthe mode of power transmission to take the form of being long, narrowand flexible.

The flexible member, which transmits the torque from an actuator at thecontroller end to the robotic manipulator, may be a cable system. Thereare two types of cable system: pulley system for cable routing andtendon-sheath system.

The tendon-sheath system generally comprises of a hollow helical coilwire acting as a sheath and a cable within that acts as the tendon. Whenthe tendon is pulled at one end, it slides within the sheath therebyallowing the pulling force to be transmitted to the other end of thesheath. The tendon in a sheath may be small enough to go through thetool channels of the endoscope and being small allows it to be easilyhandled. The difference between cable routing and tendon-sheathactuation is illustrated in FIG. 2. As compared to cable routing used inmany robots such as Utah/MIT hand and CT arms where a pulley and aplanned route for the tendon to go are required, the tendon-sheathactuation has an advantage when there are unpredictable bends inside thehuman GI tract. Other advantages of tendon-sheath actuation areflexibility and biocompatibility as compared to cable routing. For thesereasons, the tendon sheath actuation may be selected as the flexiblemember for power transmission.

The size of the robotic manipulator provides the advantage of it beingused with commercially available Guardus overtube and the like to accessthe GI tract of the animal or human body. This makes it easier, saferand more comfortable for the robotic manipulator to be introduced andremoved from the animal or human patient. In particular, the tendon maybe spectra fibres and sheath may be a helical metal coil. The spectrafibre may be bent without kinking as shown in FIG. 3 thus reducingundesirable effects such as stick and slip and sudden jerking motions atthe ends of the robotic manipulator in the human/animal body. Thehelical metal coil provides the benefit of resisting compression, notcollapsing onto the tendon, reducing friction within the roboticmanipulator and thus reducing the amount of heat produced and the wearand tear of the robotic manipulator.

The tendon and sheath may be surrounded by an overtube which protectsthe oesophagus from unintentional scratching by the robotic manipulator.The overtube may be flexible for all the sheaths. Due to the size of theovertube and the combined stiffness of the sheath within, it mayrestrain the buckling of the sheaths for better performance of thesurgery. Although tendon-sheath actuation is preferred, other forms ofactuation may be used such as, for example, signal cables to actuatorsat the distal end, and so forth. Variations may depend on the procedurerequired. For a simple procedure, one arm may be sufficient.

The arm may be configured to have a degree of freedom allowing forwardand backward motion of the arm substantially parallel to thelongitudinal axis of the endoscope. This allows for the arm to work onthe flesh in front of the endoscope without the endoscope having to bein contact with the flesh. Due to this motion, the endoscopist onlyneeds to show the view of the site where the robotic manipulator, needsto operate and does not have to adjust the distance from the site. Therobotic manipulator may also open and close itself in order to providetriangulation during the procedure as well as reducing its size when therobotic manipulator is introduced into the human body to reduce thediscomfort experienced by the patient.

The arm may comprise at least four degrees of freedom. This may resemblethe human wrist bending and rotation movement thus enabling easyemulating of the physical movement at the controller end onto the arm ofthe robotic manipulator when performing the procedure. A high number ofdegrees of freedom on the arm may be necessary to be able to performcomplicated procedures. The arm may be designed after a simplified humanarm to be as intuitive to control as possible. The arm may comprise 2 or3 degrees of freedom depending on the complexity of the surgery beingperformed.

Each degree of freedom may be controllable by two antagonistic tendonsof the flexible member. In particular, each antagonistic tendon may beindependently attachable to a motor at the controller. This design maybenefit the robotic manipulator, as the amount of rotation of eachdegree of freedom may only be dependent on its own tendon and notdependent on the movement from the other degrees of freedom thuspreventing unnecessary and unintentional movements of the arm. Therotational displacement of each joint of the arm may be directlyproportional to the linear displacements of the tendon. This allows therobotic manipulator to be controlled easily.

The number of degrees of freedom of each arm may be equal to the numberof degrees of freedom at the controller. Due to the similarity betweenthe degrees of freedom of the controller and the arm, the user caneasily visualize the control of the controller that maps to the movementof the arm. This may allow the user to use the robotic manipulator for aprolonged period of time without feeling tired.

In one embodiment, the robotic manipulator may comprise two arms. Thearms may be bundled up closely together with the endoscope. Foreffective manipulation, the arms may spread themselves out first beforemoving in to the targeted area to perform the operation. Therefore, thearms may be at least two limb lengths to prevent collision between thetwo arms and prevent blocking of vision from the endoscope.

The arms may have end effectors in the form of monopolar or bipolarelectrodes to carry out procedures involving cautery. However, becausedifferent surgical procedures require different tools, the end effectorsare interchangeable. The arm may have an end effector selected from thegroup consisting of a gripper, hook, monopolar electrocautery hook,pincer, forceps or knife. The two arms may be used to perform differentactions such as pulling and cutting of polyps or potentially suturing onthe walls of the bleeding sites. For example, the endoscopist mayconfidently use one of the end effectors to pinch onto the GI wall whilethe other holds onto a needle to perform suturing. The vision providedby the endoscope may be similar to a first person view on thesurrounding and the two arms may resemble the two human arms of thesurgeon. This gives the impression to the surgeon that he is operatinginside the patient body with his two hands allowing the surgery to bemore accurate and precise.

In particular, the end effector of one arm may be in the form of agripper, and the arm has a first joint providing a degree of freedom forcontrolling the opening and closing of the jaws of the gripper. More inparticular, the first joint may provide a further degree of freedom forcontrolling the flexion or hyperextension of the gripper. Even more inparticular, the arm may include a second joint providing a degree offreedom for controlling the supination or pronation of the arm. Thesupination/pronation joint makes it easier for the robotic manipulatorto orientate itself to perform the intended procedures. The arm mayinclude a third joint providing a degree of freedom for controlling theopening and closing of the arm, the opening of the arm being a movementout of alignment with the longitudinal axis of the endoscope, and theclosing of the arm being a movement to bring the arm into alignment withthe longitudinal axis of the endoscope. These degrees of freedoms allowthe robotic manipulator to have triangulation by making the arms spreadout from the base with the opening and closing joint before the tip ofthe manipulator closes in with the flexion/hyperextension joint. In thismanner, the arms of the robotic manipulator do not block the endoscopicview excessively so that the operation environment could be clearlyviewed by the surgeon. These degrees of freedoms also allow the roboticmanipulator to be straightened be become relatively less obstructivewhen it is being inserted into the patient.

The other two translation joints of the two arms are able to slide alonga semi-circular cap attached to the endoscope. These two translationjoints may be non-motorized joints which are controlled manually, likeconventional endoscopic tools. These translation actions are manipulatedby the pushing or pulling actions of the endoscopist intra-operatively.Once all the sheaths on the gripper side are pushed, the gripper may beable to be pushed further out to grab onto tissue. On the other hand,when pulling onto tissue, it creates a tension on the grasped tissue foreasy cutting with the cautery hook, for example.

The end effector of another arm may be in the form of a cauterizinghook. In particular, the arm may include a first joint providing adegree of freedom for controlling the flexion or hyperextension of thecauterizing hook, a second joint providing a degree of freedom forcontrolling the supination or pronation of the arm, and a third jointproviding a degree of freedom for controlling the opening and closing ofthe arm, the opening of the arm being a movement out of alignment withthe longitudinal axis of the endoscope, and the closing of the arm beinga movement to bring the arm into alignment with the longitudinal axis ofthe endoscope.

The robotic manipulator may comprise two arms comprising joints withnine degrees of freedom, wherein the first arm has an end effector inthe form of a cauterizing hook and the second arm has an end effector inthe form of a gripper.

The arm and/or the end effector may comprise a biosensor or a forcesensor or haptics figured to provide a signal to the controller. Inparticular, the force sensor may be used to give the endoscopist tactilesensations during the operation. The biosensors may enable theendoscopist to know the pH or the presence of certain chemicals at theoperating site. This will allow the endoscopist to vary the surgery tosuit the results of the sensors.

The elongation and force at the end effector may be predicted by an endeffector force prediction unit at the controller. The force predictionunit may comprise

-   -   a receiver capable of receiving information from the end        effector wherein the information allows        measurement/determination of specific parameters related to the        elongation and force at the effector end;    -   a processor capable of analysing the parameters to determine the        specific equation between the force applied at the controller        and the elongation and force applied at the end effector; and    -   a module capable of implementing the equation at the controller        to predict the force applied at the end effector of the robotic        manipulator.

This force feedback could be used to rely useful information about theprocedure back to the surgeon. A method of force prediction of thetendon sheath mechanism utilising theoretical modeling of thecharacteristic of the tendon sheath mechanism to predict the distalforce and elongation during the various phases of the actuation may beused at the force prediction unit. This force prediction method removesthe need for sophisticated sensors at the end effectors for the roboticmanipulators that have to be sterilised before use thus simplifying theprocedure and maintaining the size of the robotic manipulator. Thisforce prediction method requires a set of external sensors located atthe actuator at the controller end. The output reading of the sensorsmay be used to predict the force experienced by the end effectors. Theresult of the force prediction is used as the input for the actuator atthe controller end in order to provide force feedback to the surgeon.

Therefore by using this force prediction method, the surgeon is ablefeel the force that the robotic manipulators are exerting on thesurroundings. This ensures the surgeon does not cause unnecessary traumato the patient's body and also ensures that the robotic manipulator orthe system does not break down due to excessive tension on the tendon.The surgery may then be carried out faster, safer and in a moreconsistent manner.

According to another aspect, the present invention provides a controllerfor controlling the movements of a robotic manipulator for flexibleendoscopy comprising:

-   -   a hand-held member configured for use by a user to effect        movements of the robotic manipulator,        wherein the hand-held member comprises joints providing degrees        of freedom corresponding to the degrees of freedom of the        robotic manipulator.

The controller may be able to generate movements above the level of theuser's detection and may provide significant information from therobotic manipulator to the user. The controller may also possess higherposition resolution than the robotic manipulator. Its friction, mass andinertia may be low enough to give comfort to the user.

The controller may include a microprocessor configured to:

-   -   detect the motions of the hand-held member,    -   scale the motion detected to suit the robotic manipulator, and    -   transmit signals to an actuator for controlling flexible members        connected to the robotic manipulator.

The microprocessor may also be a motion controller. In one exemplaryembodiment, it is a console and is essentially the ‘brain’ of thesystem. It reads information from the controller. Software calculatesthe required kinematics of the robotic manipulator. Output signals arethen sent to the motion controller to actuate the motors and other primemovers accordingly. Input signals from the robot manipulator's sensorsare also constantly being read by the microprocessor to ensure that theformer is moving in the required manner. Other functions of thismicroprocessor include scaling down of the clinician's movements.Ideally, the robotic manipulator should move by significantly less thatthe movement of the clinician movement for accuracy and safety.

The controller may comprise multiple rotary encoders that read thevalues of the movement the user makes and feeds it to the microprocessorfor analysis and subsequently control of the robotic manipulator.

The hand-held member may include a prime mover to receive signals fromthe robotic manipulator and to provide feedback to a user using thehand-held member. This allows the user to feel a sensation when therobotic manipulator exerts a force on the environment during theoperation. This makes the operation safer and faster due to theadditional information.

The microprocessor may comprise a computer that does the mapping betweenthe readings of the controller as well as electronic housing that hasall the relevant wirings for the system to work. The microprocessorreads in the sensors reading from the rotary encoders of the controllerand then uses a software program that scales and actuates the amount ofmovement of the various prime movers at the actuator.

The microprocessor may comprise an electronics housing to hold all therelevant circuitries of the system such as amplifiers and power suppliesand protect them from the outside elements and ensures the wiring withinis not disturbed during transportation to prevent any wiring errors tothe system.

The actuator may be designed to be easily portable as well as a compactsize.

The hand-held member may comprise grippers attachable to fingers of theuser. In particular, the robotic manipulator may be controllable usingthe grippers. That means the user can rest his/her elbow, improving thecomfort and user-friendliness of the controller. The controller mayfurther comprise an armrest configured to receive a user's arm forproviding greater comfort.

The hand-held member may comprise a plurality of linkages that may beadjustable to suit different users. The size of the linkages may besmall so that the weight of the controller may be reduced.Counter-weight mechanisms may be put in place to ensure that the userfeels very little weight when operating the controller. Also, thenon-wearing characteristic of the controller eliminates the weight oflinkages that would help the surgeon control the controller effectivelywhen the operation is carried out for a long time. This reduces userfatigue. The length of the base of the gripper linkage may beadjustable. This makes it possible for motion scaling mechanically, ifnecessary. Vision intrusion may also be reduced with simpler and fewerlinkages.

Two different-colour pedals at the base of the controller may be presentto control the cauterization and coagulation mode of the hook. Whilefocusing on the task and being busy with the controller on hand, it maybe more convenient for the surgeon to control the cutting action bystepping onto the pedals with his foot.

According to another aspect, the present invention provides a roboticsystem for flexible endoscopy comprising a robotic manipulator accordingto any aspect of the present invention and a controller according to anyaspect of the present invention.

According to a further aspect, the present invention provides a methodof flexible endoscopy comprising the step of inserting the roboticmanipulator according to any aspect of the present invention into anatural orifice of the human body.

According to one aspect, the present invention provides a method oftreatment of a gastrointerinal tract related disease comprising the stepof inserting the robotic manipulator according to any aspect of thepreset invention into a natural orifice of the human body.

According to another aspect, the present invention provides a use of arobotic system according to any aspect of the present invention for thetreatment of a gastrointerinal tract related disease.

According to yet another aspect, the present invention provides a methodof predicting the force and elongation at the end of a roboticmanipulator, comprising the steps of:

-   -   (a) receiving information from the end of the robotic        manipulator, wherein the information allows        measurement/determination of specific parameters related to the        elongation and force at the end of the robotic manipulator,    -   (b) analysing the parameters to determine the specific equation        between the force applied at a controller and the elongation and        force applied at the end of the robotic manipulator; and    -   (c) implementing the equation at the controller to predict the        force applied at the end effector of the robotic manipulator

This method may be used for predicting the end effector parameters forany robotic arm having relatively constant sheath shapes. This offers anadditional advantage of locating the sensors away from the end effector.This reduces the inertia of the end effector and allows the roboticmanipulator to work in extreme environmental conditions, whereby sensorsat the end effector would fail.

As will be apparent from the following description, preferredembodiments of the present invention allow an optimal use of bothrobotic manipulator operations and controller operations to takeadvantage of the manoeuvrability and size of these components inflexible endoscopy to carry out dextrous GI related procedures moreefficiently and accurately. This and other related advantages will beapparent to skilled persons from the description below.

BRIEF DESCRIPTION OF THE FIGURES

Preferred embodiments of the invention will now be described by way ofexample only with reference to the following figures:

FIG. 1 is a schematic layout of the exemplary embodiment of the roboticsystem (Master And Slave Transluminal Endoscopic Robot; MASTER);

FIG. 2 is a diagram showing the difference between cable routing andtendon-sheath actuation;

FIG. 3 is an image showing the buckling of the sheath;

FIG. 4A is a diagram showing the overtube with the loaded sheaths, andFIG. 4B illustrates a tendon within a helical metal coil;

FIGS. 5 A and 5B are views of the exemplary embodiment roboticmanipulator showing the size of the robotic manipulator;

FIG. 5C is a perspective view of the exemplary embodiment roboticmanipulator attached to an endoscope;

FIG. 6 is a perspective view showing the degrees of freedom of theexemplary embodiment robotic manipulator;

FIG. 7 is a perspective view of the robotic manipulator pointing out thejoints and the parameters of each joint;

FIG. 8 is an exploded view of the exemplary embodiment roboticmanipulator;

FIG. 9 is a diagram showing the similarity in design of the exemplaryembodiment robotic manipulator with the human arm from the wrist to theelbow;

FIG. 10 is a diagram showing the workspace of the exemplary embodimentrobotic manipulator;

FIG. 11 is a view of an exemplary embodiment actuator for use with thesystem of FIG. 1;

FIG. 12 is a view of an exemplary embodiment controller for use with thesystem of FIG. 1;

FIG. 13 is a plan view of the exemplary embodiment of a controller whenin use;

FIG. 14 is a diagram of the ball and socket joint of the exemplaryembodiment controller;

FIG. 15 is a view of an exemplary embodiment of an electronics housingof the controller;

FIG. 16 is an image showing the vision from an endoscope duringendoscopic submucosal dissection (ESD);

FIG. 17 is a graphical representation of submucosal dissection times fora consecutive series of five trial ESD procedures in pigs;

FIG. 18 is an image of an excised gastric lesion;

FIG. 19 is a picture of the ex vivo test during grasping and cuttingprocess;

FIG. 20 is a model of a small section dx of a tendon and sheath;

FIG. 21 is a simplified model of the sheath compared with a genericsheath;

FIG. 22 is a graphical representation of M_(e) and K;

FIG. 23 is a graphical representation showing the gradually reducingT_(in) and the tension distribution within the sheath;

FIG. 24 is a graphical representation showing the gradually increasingT_(in) and the tension distribution within the sheath;

FIG. 25 is a simplified model showing the limit of motion for a simplerevolute joint;

FIG. 26 is a graphical representation showing the phases of pulling andreleasing that is studied;

FIG. 27 is a graphical representation showing the result of the actualforce/elongation vs predicted force/elongation.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the exemplary embodiment shown in FIG. 1, the robotic systemcomprises a controller 300 able to be operated by an endoscopist 402optionally with the help of an assistant 400. The robotic system alsohas a microprocessor 500 connected to the controller 300 to controlmovements of a robotic manipulator 100 on a patient 404. The systemfurther comprises a conventional endoscopy system 406.

The robotic manipulator 100 comprises a flexible member (a tendon insheath in the preferred embodiment) configured to be coupled to anendoscope, and an arm connected to and movable by the flexible member.As will be detailed below, the flexible member has a first end connectedto the arm and a second end connectable to a controller to allow aphysical movement of the arm to be controllable by a physical movementof the controller.

FIG. 4A shows an exemplary embodiment of a plurality of tendons 110 insheaths 112 as a flexible member. To allow the endoscope 406 and therobotic system to be inserted with ease, an overtube 111 may be firstinserted through the oesophagus. The overtube 111 tightly constrains thesheaths 112 and prevents buckling. The overtube 111 itself is stillhighly flexible for the application of surgical robot. FIG. 4B shows atendon 110 within a sheath 112 implemented as a helical metal coil.

FIGS. 5A and B shows an exemplary embodiment of the robotic manipulator100. The robotic manipulator 100 is able to operate with the requirednumber of degrees of freedom (DOFs) to accurately replicate the hand andwrist motions of the endoscopist 402 within the GI tract in real time.The robotic manipulator 100 includes a flexible member 102 configured tobe coupled to an endoscope 406 with a first end (that enters thepatient) of the flexible member connected to an arm 104 a, 104 b and thesecond end connectable to a controller (not shown). In order to avoidinterference with the operation of the endoscopist, the length of theflexible member 102 is 2 m, which is 0.5 m longer than the endoscope.The exemplary embodiment of the robotic manipulator 100 has two arms 104a, 104 b. One end of the arms is attached to the flexible member 102 andthe other end of the arms is attached to an end effector 103. The arm104 a is attached at one end to an end effector 103 in the form of agripper 106. The arm 104 b is attached at one end to an end effector 103in the form of a hook 108.

FIG. 5C shows an exemplary embodiment of the robotic manipulator 100coupled to an endoscope 406. As noted earlier, the flexible member 102comprises a plurality of tendons 110 in sheaths 112.

The endoscope 406 has a tool channel (not shown) into which the flexiblemember 102 can be inserted to drive a robotic manipulator 100 witheffector ends 103. The endoscope 406 is inserted by the endoscopist 402who can observe progress on a monitor (not shown). When the endoscope406 has traversed to the area of interest within the GI tract, therobotic manipulator 100 is inserted by the clinician 402 until the endeffectors 103 appear at the distal end of the endoscope 406. Theclinician 402 then moves to the controller 300 where he uses his fingersto control an ergonomically designed mechanical controller 300 as isdescribed below. The entire robotic manipulator 100 is designed to besmall, slender and flexible enough to be threaded through the toolchannels of a dual-channel therapeutic endoscope 406 (GIF-2T160, OlympusMedical Systems Corporation, Japan), which is connected to a standardendoscope image system (EVIS EXERA II Universal Platform, OlympusMedical Systems Corporation, Japan).

Ideally, the system would be operated by an endoscopist and a surgeon.The former would traverse and manoeuvre the endoscope while the latterwould sit in front of the controller to control the roboticmanipulators.

The exemplary embodiment robotic manipulator 100 illustrated has ninedegrees of freedom (indicated with arrows in FIG. 6) and isanthropomorphic to the human arm (elbow to wrist). A robotic manipulator100 with two arms 104 a, 104 b attach to the distal end of aconventional flexible endoscope 406 using an attachment 105 that couplesto the distal end and supports the arms 104 a, 104 b. The arms 104 a,104 b together with the end effectors 103 are actuated by flexiblemembers 102 connected to motors (not shown) at the proximal ends (i.e.ends connected to an actuator).

In order to actuate one degree of freedom, two sets of tendons 110 andsheaths 112 which work antagonistically are required for a joint to movebidirectionally. Two tendon 110 and two sheath 112 cables control onemotorized degree of freedom of the robotic manipulator. At both proximaland distal ends, the sheath 112 will be stopped at the counter bore holeof the base of each degree of freedom while the tendon 110 is slidthrough that counter-bore hole and a tiny hole on the pulley or therotational body. To control one degree of freedom, two tendons 110 haveto be clamped at the pulley side of the robotic manipulator with wirefittings.

The kinematic of the robotic manipulator is represented by theDenavit-Hartenberg (DH) parameters which joint configuration andparameters of each joint are depicted in FIG. 7, and Table 1,respectively. In addition, the resultant homogeneous transformationmatrix is as follow where cθ_(i)=cos(θ_(i)), sθ_(i)=sin(θ_(i)).

$T:=\left\lbrack \begin{matrix}{\begin{pmatrix}{{{- s}\; {\theta 1s}\; {\theta 2}} +} \\{c\; {\theta 1}\; c\; {\theta 2}}\end{pmatrix}c\; {\theta 3}} & \begin{matrix}{{{- c}\; \theta \; 1s\; {\theta 2}} -} \\{s\; {\theta 1}\; c\; {\theta 2}}\end{matrix} & {\begin{pmatrix}{{{- s}\; {\theta 1}\; s\; {\theta 2}} +} \\{c\; {\theta 1}\; c\; {\theta 2}}\end{pmatrix}s\; {\theta 3}} & \begin{matrix}{{\begin{pmatrix}{{{- c}\; {\theta 1}\; s\; {\theta 2}} -} \\{s\; {\theta 1}\; c\; {\theta 2}}\end{pmatrix}d\; 3} +} \\\begin{matrix}{{c\; {\theta 1}\; a\; 2c\; {\theta 2}} -} \\{s\; {\theta 1}\; a\; 2s\; {\theta 2}}\end{matrix}\end{matrix} \\{\begin{pmatrix}{{s\; {\theta 1}\; c\; {\theta 2}} +} \\{c\; {\theta 1}\; s\; {\theta 2}}\end{pmatrix}c\; {\theta 3}} & \begin{matrix}{{{- s}\; \theta \; 1s\; {\theta 2}} +} \\{c\; {\theta 1}\; c\; {\theta 2}}\end{matrix} & {\begin{pmatrix}{{s\; {\theta 1}\; c\; {\theta 2}} +} \\{c\; {\theta 1}\; s\; {\theta 2}}\end{pmatrix}s\; {\theta 3}} & \begin{matrix}{{\begin{pmatrix}{{{- s}\; {\theta 1}\; s\; {\theta 2}} +} \\{c\; {\theta 1}\; c\; {\theta 2}}\end{pmatrix}d\; 3} +} \\\begin{matrix}{{s\; {\theta 1}\; a\; 2c\; {\theta 2}} +} \\{c\; {\theta 1}\; a\; 2s\; {\theta 2}}\end{matrix}\end{matrix} \\{{- s}\; {\theta 3}} & 0 & {c\; {\theta 3}} & {{d\; 2} + {d\; 0}} \\0 & 0 & 0 & 1\end{matrix} \right\rbrack$

Based on the calculated DH parameter matrix, the parameters of each linkand joint, the workspace formed by the two arms of the roboticmanipulator is depicted in FIG. 10. The grey spaces are the workspace ofthe tip of each manipulator. With such design, the robot could perform avariety of complex tasks with minimal blockage to the endoscopic cameraview.

The design of the controller (as explained below) is anthropomorphic andreplicates the degrees of freedom of the robotic manipulator. However,the achievable workspace for the whole system depends on the range ofhuman arm motion as represented in Table 2. The workspace is formedusing the motion range of movement of the joint from −90° to 90°although the robotic manipulator can move beyond this range.

TABLE 1 DH Parameters of robotic manipulator Joint (i) θ(i) A(i-1)a(i-1) d(i) 1 0    0 0 d0 2 θ1    0 0 0 3 θ2 −90 a2 d2 4 θ3   90 0 d3

TABLE 2 Allowable range of motion of the controller and roboticManipulator Range Robotic No: Motion Manipulator Controller 1Translation 15 mm 2 Opening/Closing −45° to 90° −45° to 45° 3Supination/Pronation −90° to 90° −90° to 90° 4 Flexion/Extension −90° to90° −90° to 90°

The force that the joint exerts on the end effectors is measured andsummarized in Table 3. The maximum grasping force at theflexion/extension joint is approximately 5.20N, which is sufficient tohold and grasp onto the slippery and viscoelastic tissue during theprocedure.

TABLE 3 The force exerted of the end effector's joint Joint NameForce(N) Opening/Closing 2.87 Supination/Pronation 3.29Flexion/Extension 5.20

FIG. 8 shows the parts that make up the robotic manipulator 100 when theend effectors are a hook 108 and a gripper 106 and FIG. 7 shows thedegrees of freedom that correspond to the exemplary robotic manipulator100 of FIGS. 5 and 6. Each robotic manipulator 100 has a right and aleft arm base 128, 140. The joint at the arm bases 128, 140 provides adegree of freedom for controlling the translation of the arms 104 a, 104b allowing forward and backward motion of the arms 104 a, 104 bsubstantially parallel to the longitudinal axis of the endoscope. Thearm bases 128, 140 of each robotic manipulator 100 are anthropomorphicto the elbow, and have two orthogonal rotational opening joints 138 a,138 b. These opening joints 138 a, 138 b provide a degree of freedom foropening and closing of the arms 104 a, 104 b, the opening of the arms104 a, 104 b being a movement out of alignment with the longitudinalaxis of the endoscope 406 (not shown), and the closing of the arms 104a, 104 b being a movement to bring the arm 104 a, 104 b into alignmentwith the longitudinal axis of the endoscope 406 (not shown). The rightarm base 128 and left arm base 140 are held in place by a cap 142 whichis held close by a cap retaining ring 126. The two orthogonal rotationalopening joints 138 a, 138 b are each independently fixed to one of thearm bases 128, 140 by M1X3 screw 124. Further along the arm 104 a, ofthe robotic manipulator 100, away from the flexible member (not shown)is the rotating base retaining ring 122 a which attaches the rotatingbase 136 a to the opening joint 138 a of the arm 104 a. The rotatingbase 136 a is connected to a gripper base 134 a. Both arms 104 a, 104 bhave a similar structure up to this point where the end effector 103 atthe ends of the arms 104 a, 104 b may vary. As shown, the end effector103 is fixed with grippers 106 at arm 104 a which are used to grabtissue. The distal tip of the other end effector 103 is fixed with ahook 108 with which monopolar, cautery and cutting can be performed.

In the exemplary robotic manipulator 100 of FIGS. 5 and 6, one arm 104 aends with a gripper 106 and the other arm 104 b ends with a cauterizinghook 108. The gripper base 134 a of arm 104 a is coupled to the gripper106 and held in place by a gripper retaining pin 120. The gripperpositioning pins 118 hold teeth tip A 116 and teeth tip B 114 at the endof the gripper 106 furthest away from the flexible member 102 in place.The joints between the teeth 114, 116 and the gripper 106, and betweenthe gripper 106 and the gripper base 134 a provide two degrees offreedom for flexion and hyperextension of the teeth 114, 116 and gripper106 respectively. The joints in the preferred form comprise a 1×3×1 ballbearing 132. The rotating base 136 a provides for a degree of freedomfor supination/pronation of the gripper 106.

The other arm 104 b of the exemplary robotic manipulator 100 has agripper base 134 b of arm 104 b coupled to the cauterizing hook 108 by ahook base 130 and a 1×3×1 ball bearing 132 b. The ball bearing 132provides a joint at the gripper base 134 b that provides for a degree offreedom for flexion and hyperextension of the cauterizing hook 108. Therotating base 136 b provides for a degree of freedom forsupination/pronation of the cauterizing hook 108.

The nine degrees of freedom for the preferred form two arms aretherefore as follows:

-   -   i. translation forward and backward of the first arm    -   ii. translation forward and backward of the second arm    -   iii. opening and closing of the first arm    -   iv. opening and closing of the second arm    -   v. supination and pronation of the first arm    -   vi. supination and pronation of the second arm    -   vii. flexion and hyperextension of the first arm    -   viii. flexion and hyperextension of the second arm    -   ix. flexion and hyperextension of the jaws of the first arm

FIG. 9 shows the similarity in design of the robotic manipulator 100with the human arm from the wrist to elbow. This similarity wasintentionally provided to simplify the robotic manipulator as well asmaking it more nimble to perform treatment on the patient. This alsomakes it less tiring for the surgeon to perform the procedure since hecan rest his shoulders on the flat surface.

The workspace of the robotic manipulator 100 can be seen in FIG. 10. Theworkspace is formed using the range of movement of the joint assimplified from −90° to 90° although the robotic manipulator 100 canmove beyond this range.

The exemplary embodiment of an actuator 200 is shown in FIG. 11 andhouses the motors, sensors and other mechatronic devices (not shown)required to actuate the robotic manipulator 100. The roboticmanipulator's 100 one end is affixed to actuator 100 by way of theflexible member 102. The actuator 200 acts upon the flexible member 102based on signals received from the controller 300.

The actuator 200 comprises a housing enclosing an electronic housing anda motor housing (not shown). The former houses the power supply,actuator amplifier, and motion controller interconnector. The latter isused to house DC motors and sheath stoppers 113, which are used tosecure the sheath and tendon. The actuator housing comprises sevenmotors for seven motorized degrees of freedom and comprises threecomponents: the front plates to secure sheaths, the side plates tosecure the actuators and the rotating drums to secure and control thetendons. The drums are attached to the motor shaft. All the plates arerestrained firmly within a structure of aluminium profiles which allowsfor easy disassembly if there is a need for repair and troubleshooting.The actuators are packed together in a tight configuration to save spacein the operating theatre.

The exemplary embodiment of the controller 300 is shown in FIGS. 12 and13. The controller 300 comprises two hand-held members 306 a, 306 b.Each member 306 is configured for use by a user to effect movements ofthe robotic manipulator 100. The hand-held member 306 comprises jointsproviding degrees of freedom corresponding to the degrees of freedom ofthe robotic manipulator 100. The controller includes a microprocessor500 configured to detect the motions of the hand-held member 306, scalethe motion detected to suit the robotic manipulator 100 and transmitsignals to the actuator 200 for controlling flexible members 102connected to the robotic manipulator 100 (not shown). The hand-heldmembers 306 include a prime mover (not shown) to receive signals fromthe robotic manipulator 100 and to provide feedback to a user 402 usingthe hand-held member 306. The hand-held members 306 comprise grippers308 attachable to fingers of the user 402. The controller 300 furthercomprises an armrest 310 configured to receive a user's arm. Thehand-held members 306 comprise a plurality of linkages 312 that isadjustable to suit different users. The controller 300 is kept within ahousing 302. As can be seen in FIG. 14, from the design of thecontroller 300, the three revolute joints intercept at one point,creating a ball-and-socket joint. The position and orientation of thegrippers 308 are determined by these three joints. Thus, the kinematicanalysis of the controller 300 is performed on just one ball-and-socketjoint.

The clinician 402 places his fingers within the finger linkages 308 ofthe hand-held members 306 and can freely move his wrists and fingers.With the vision system, the clinician 402 would be able to see therobotic manipulator protruding from the endoscope's distal tip. Themovements of the robotic manipulator would be in strict accordance tohow the clinician 402 manipulates the hand-held members 306. Thehand-held members 306 are embedded with an array of linear and rotaryencoders which sense the orientation of the clinician's wrists andfingers (fingers being taken to include the thumbs). This information isfed into the microprocessor 500 for further processing. Themicroprocessor 500 processes the received data that follow by sendingcommands to the actuator housing 200 to control the tendon-sheathactuation 102. Based on the force prediction modeling, the controllerhas also been implemented with actuators, which are able to provideforce feedback to the surgeon onto two selected degrees of freedom,namely the opening and closing joints. Some or all of the joints of thedevices 306 may be connected to motors which would exert resistingforces on the clinician's hand movements. This mechanical featureenables the clinician to have a force feedback during the operation. Assuch, the wall of the GI tract can be ‘felt’ by the clinician when theend effector 103 comes in contact with it. The hand-held members 306have the nine rotational degrees of freedom, and all of the angulardisplacements may be sensed by rotary encoders.

The system according to this invention include the microprocessor 500 asshown in FIG. 15, which comprises a computer (not shown) that does themapping between the readings of the controller 300 as well as anelectronic housing 502 that comprises all the relevant wirings 504 forthe system to work such as amplifiers and power supplies and protectthem from the outside elements.

The systems, devices and methods of the various exemplary embodiments ofapplicant's robotic system in tandem with current flexible endoscopes.Because the surgical procedures would be performed through naturalorifices, the systems, devices, and methods of applicants robotic systemcan perform what may be characterized as “no-hole surgery”, which isless invasive than key-hole surgeries.

The robotic system may be used for procedures other than those of the GItract. It may be used for any surgical procedure able to be performedwith flexible scopes. These include appendectomy (removal of appendix),removal of gall bladder, tying of fallopian tubes, and so forth. Therobotic system may give the surgeon more dexterity and manoeuvrability.

EXAMPLES Example 1

Each ESD in live animal was repeated using the conventional endoscope.The main outcome measures were: (i) time required to complete thesubmucosal dissection of the entire lesion, (ii) dissection efficacy,(iii) completeness of the excision of the lesion, and (iv) presence orabsence of perforation of the wall of the stomach.

Submucosal dissection time was defined as the time from activation ofthe endoscopic dissecting instrument to completion of excision of theentire lesion. Assessment of dissection efficacy was based on scoring ofefficiency of two related task components—grasping and cutting oftissue—on a graded structured scale from 0 to 2, where the lowest grade,“0” means failure to grasp/cut and the highest grade “2” means a mostefficient grasp/cut. Similarly, the completeness of the lesion excisionis rated on a scale of 0 to 3, where “0” means failure to excise and “3”means a complete excision of the targeted lesion in one single piece.The details of the structured grading system are shown in Table 4.Grading was done by the operator and recorded on the spot. Presence ofany inadvertent perforation of the stomach wall was checked by the airleak test in the Erlangan models and endoscopic visual inspection in thelive animals.

TABLE 4 Structured grading system for assessing dissection efficacy andcompleteness of excision Dissection Efficacy Score Grasping of tissueFailure to grasp tissue 0 Able to grasp but unable to apply tissuetraction 1 Able to grasp and retract tissue with some degree of tension2 Cutting of tissue Tissue is not cut at all at standard power setting(80 W) 0 More than 3 attempts to cut tissue at standard power setting(80 W) 1 Less than 3 attempts to cut tissue at standard power setting(80 W) 2 Completeness of excision Failed excision 0 Excision of lesionin more than 3 pieces 1 Excision of lesion in less than 3 pieces 2Excision of lesion in 1 single piece 3

For operation, the robotic system (Master And Slave TransluminalEndoscopic Robot; MASTER) comprised the dual-channel therapeuticendoscope (GIF-2T160, Olympus Medical Systems Corporation, Japan)connected to a standard endoscopy platform (EVIS EXERA II UniversalPlatform, Olympus Medical Systems Corporation, Japan) withhigh-definition visual display and real-time video recording functions.An attached electrical surgical generator regulated and monitored thepower output used for the monopolar resection (cutting and coagulation).Operation was conducted through the ergonomically designed steerablemotion sensing controller with two articulating arms (FIG. 5). Thecontroller was embedded with an array of linear and rotary encoders. Tooperate, the operator simply fits his/her wrists and fingers into thetwo articulating arms and moves them in the same way he/she would tomanipulate the end-effectors directly. Motions are detected by the arrayof sensors and actuated into force signals to drive the manipulator andend-effectors via a tendon-sheath mechanism. This allows the operator tointuitively control the operation remotely. The controller and therobotic manipulator are both equipped with nine rotational degrees offreedom.

The reference system used was a standard conventional therapeuticendoscope (GIF-2T160, Olympus Medical Systems Corporation, Japan) withthe usual accessories such as the insulation-tipped (IT) diathermicknife (Olympus Medical Systems Corporation, Japan) and injectionneedles.

Five Erlangen porcine stomachs and five pigs aged between 5 to 7 monthsold, and each weighing about 35 kg were used and ESD was first performedon the Erlangan models, and then on each of the live pigs, consecutivelyusing the robotic system of the present invention (MASTER). Forcomparison purpose, the ESD procedures in the same 5 live pigs wererepeated using the conventional endoscope.

Fresh porcine stomach was mounted on a specially designed dissectionplatform to simulate its normal orientation in the body. A standarddual-channel therapeutic endoscope was then passed into the stomachthrough an overtube and the stomach was flushed with normal saline.Using the IT diathermic knife, artificial gastric lesions approximately20 mm in diameter were marked by means of spotty cautery on the mucosaof the cardia, antrum and body of the stomach. Before the ESD, each ofthese lesions was elevated by submucosal injection of a cocktail of 40ml normal saline and 2 ml of 0.04% indigo carmine. The conventionalendoscope was then removed and a dual-channel therapeutic endoscope withthe robotic system of the present invention mounted was introduced intothe stomach. Using the robotic grasper to hold the elevated lesion, aperipheral mucosal incision was made using the monopolar electrocauteryhook (Power setting, 80 W) at a circumferential margin of 1 cm from thedemarcated area. Once completed, the mucosal flap was lifted using thegrasper. Cutting line visualization was maintained as the monopolarelectrocautery hook was applied underneath the flap in a directionparallel to the muscle layer to cut the lesion through the submucosalplane. Dissection was executed in a single lateral direction untilcompletion and the entire lesion was excised enbloc. On completion ofthe experiment, the stomach was insufflated with air to detect for leakdue to any inadvertent perforation caused during ESD. The stomach wasthen cut and opened for inspection of completeness of lesion resection.

For study in live animals, the pig was food deprived for 18 hrs justbefore the procedure. The animal was sedated for pre-surgicalpreparation. A preanesthetic cocktail of ketamine 20 mg/kg and atropine0.05 mg/kg intramuscularly was adminstered, following which anesthesiawas induced with 5% intravenous isoflurane. The animal was thenintubated with an endotrachael tube. General anesthesia with 1-2%isoflurane followed. Throughout the operation, oxygen was administratedto the animal at a flow rate of 2.0 litre/minute, while heart rate andSpO2 were monitored every 20 minutes.

As with the Erlangan models, gastric lesions were marked using the ITknife and lesions were elevated by a submucosal injection of a cocktailof 40 ml normal saline and 1 ml indigo carmine (40 mg/5 mL). ESD wasperformed in a similar manner as described under the method for Erlanganmodels and the entire lesion was excised enbloc and retrieved throughthe mouth. Hemostasis, where necessary, was achieved with theelectrocautery hook (Power setting, 60 W). In each animal, theexperiment was repeated on one other lesion using the conventionalendoscope, in which case, an IT knife was deployed through the accessorychannel of the endoscope to do the dissection. After the operation, thestomach was visually inspected for signs of perforation. Once done, theanimal was euthanized according to IACUC approved protocol.

Data Capture and Analysis

During the experiment, an independent assessor recorded the sequence andtiming of all pertinent tasks. The collected data was entered into anExcel spreadsheet and data was analyzed using simple descriptivestatistics. The average time taken to dissect the entire lesion inErlangen models and live animals were separately computed. The meandissection time taken by the MASTER was compared to that taken byconventional endoscopy method using simple student's t-test.

Results

In the study on Erlangen porcine stomach models, a total of 15 gastriclesions located at the cardia, antrum, or body of the stomach weresuccessfully excised in single piece following submucosal dissectionusing the MASTER, with no incidence of gastric wall perforation. Themean dimension of the excised specimens was 37.4×26.5 mm. Meansubmucosal dissection time was 23.9 min (range, 7-48 min). There was nosignificant difference between the dissection times of lesions at thedifferent locations in the stomach (P=0.449).

In the experiment on five live pigs, the MASTER took a mean of 16.2 min(range, 3-29 min) to complete the submucosal dissection (FIG. 16). Thedissection time was 29, 18, 19, 12, 3 min for the consecutive series offive animals, respectively, in the order they were performed (FIG. 17).This compares with a similar mean dissection time of 18.6 min (range,9-34 min) taken by the conventional endoscopic system with an IT knifedeployed through its accessory channel (P=0.708). The dissection timefor ESD performed using the conventional method in five consecutiveanimals was 9, 23, 34, 15, 12 min, respectively. In both series of ESDsperformed, all lesions were efficiently excised enbloc. The meandimension of the specimens resected by the MASTER was 37.2×30.1 mm;those resected by conventional endoscopy with IT knife averaged32.78×25.6 mm. A sample specimen is shown in FIG. 18. There was noincidence of excessive bleeding or gastric wall perforation in eithergroup of animals.

In all the experiments conducted using the MASTER, control of therobotic manipulator was easily achieved by the operator steering theergonomically designed motion sensing controller. Triangulation of thetwo arms was achieved with ease and robotic coordination of the twoend-effectors was precise. Enbloc resection of all lesions was easilyperformed by the robotic monopolar electrocautery hook cutting set at apower of 80 W, attaining good cutting efficiency (Grade 2) throughout.In all cases, it took the operator less than three attempts to cutthrough the gastric submucosa. Grasping and retraction tension of therobotic grasper performed just as well (Grade 2); the operator couldgrasp and retract tissue with some degree of tension throughout theprocedure. All surgical maneuvers were accurate; end-effectors' aimswere on target all the time and no untoward incident such as injury tosurrounding tissue and vasculature occurred.

It is demonstrated for the first time in live animals how MASTER canmitigate some of the fundamental technical constraints in endoscopicsurgery to facilitate performance of ESD, a technically challengingendoluminal procedure. MASTER represents a breakthrough deconstructionof the endoscopy platform, by introducing robotic control of surgicaltools and tasks through an ergonomic human-machine interface builtaround the original endoscopic paradigm. It separates control ofinstrumental motion from that of endoscopic movement such that surgicaltasks may be independently executed by a second operator via ahuman-machine interface. With it, endoscopically deployed instrumentscan be independently controlled, allowing thus bimanual coordination ofeffector instruments to facilitate actions such as retraction/exposure,traction/countertraction, approximation and dissection of tissue.Robotic technology increases the degrees of freedom for mobility ofendoscopic instruments deployed at the distal end of the endoscope. Withnine degrees of freedom at the manipulating end of the roboticmanipulator, MASTER allows the operator to position and orient theattached effector instrument at any point in space. This enablestriangulation of surgical end-effectors otherwise not possible withstandard endoscopy platforms. Through the master-slave system,significant force could be exerted to the point of action, allowing theend-effectors to effectively manipulate and dissect the tissue, as inthe submucosal dissection in the present performance of ESD.

MASTER has clear advantages over standalone surgical robots as it is notas bulky and is designed to be adaptable to any standard dual channelendoscope. It requires a minimum of just two operators to perform anendoscopic surgery, just as in the performance of the ESD we justdescribed. With the precision and efficiency of the MASTER, the entireESD operation could potentially be completed in a very short time.Although in this pilot trial, no significant difference was seen in themean submucosal dissection times taken by MASTER and the conventionalendoscope with IT knife, it is believed the system could perform betterand faster once operators become more accustomed to its use. Thispreliminary evaluation of MASTER for endoscopic surgery is limited inthe sense that operators have yet to fully master the operation of thenew equipment. The present performance results reflect just the earlypart of a learning curve. In the present series of live animal studies,the first dissection took 29 minutes, but the dissection timesubsequently dropped to 3 min at the final or 5^(th) procedure. Asoperation of the MASTER is intuitive, it is not difficult for a noviceto master the skills. Implementation of its use in endoscopic surgerywill therefore not require as long a learning curve as with conventionalendoscopy systems. Despite this being the initial trial application ofthe MASTER, no untoward injury to surrounding organs, tissue orvasculature occurred.

MASTER is a promising platform for efficient and safe performance ofcomplex endoluminal surgery such as ESD. It is expected that withfurther developments such as refinement of the system, incorporation ofhaptic technology for tactile and force feedback, and addition ofadaptable auxiliary devices, as well as a complete armamentarium ofuseful swappable end-effectors, the functionality of the endosurgicalsystem would be greatly improved and expanded to adequately support bothendoluminal and transluminal surgery.

Example 2

Before the experiment was performed on living animals, MASTER was firsttested on explanted porcine's stomach. The main objective of the ex vivoexperiment was to test the capability of the system in grasping andcutting performance. The grippers must provide enough force for graspingalong with manipulating the tissue while the hook must be able toperform the cut at the desired site of the tissue. The test alsoestablishes the teamwork and the cooperation between the endoscopist whohas more than 20 years of clinical experience and the surgeon whocontrols the controller with less than 5 years of experience. With 15times of training with explanted tissues, the result showed thefeasibility of the system before being conducted in real animal.

The liver wedge resection procedure was chosen to test the feasibilityof the system to perform NOTES. The in vivo test was performed at theAdvance Surgical Training Center, National University Hospital inSingapore with the help of experienced endoscopists and surgeons. Usingthe controller and robotic manipulator manipulator was successfully usedto perform two in vivo liver wedge resections on animals through NOTESprocedure. To perform the liver wedge resections, the manipulatorfirstlyperformed gastrotomy, in which an incision is made from within thestomach to access the peritoneal cavity of a live pig. Once the robot isinside the peritoneal cavity, the endoscopist would control theendoscope to reach the liver side to perform the liver wedge resection.The grippers of the manipulator then held onto the edge of the liverwhile the electrocautery hook proceeded to cut out a piece of the liver.During this procedure, the grippers had to grasp the edge of the liverfirmly to provide tension for the cauterizing cut to be effective. Theliver resection process took approximately 9 minutes for each of the twoin vivo animal trials. After the liver wedge resection was perform, withthe grasper still holding onto the resected tissue, the hook was stillfree to perform haemostasis at the freshly cut portions of the liver toarrest bleeding. The surgeon then removed the robot from the porcine andretrieved the liver wedge for analysis. The dimensions of the liverpieces taken out are shown in Table 5. Two trials were merely performeddue to stringent regulation, however to further justify the performancesof the system; more animal trials would be conducted in the future.

TABLE 5 The time required and the size of the liver wedge resection Time(mins) Length (mm) Width (mm) Pig 1 8.5 21 10 Pig 2 8.2 14  8

Ex vivo and in vivo experiments were conducted. Two liver wedgeresections were performed successfully with the MASTER system. Theresults showed the potential of the system to be implemented in otherapplications of NOTES such as removal of the appendix and gall bladder.

In the near future, the size of the robotic manipulator would be furtherreduced to enable the change of the end effectors intraoperatively Theforce feedback will be evaluated and applied to the rest of the degreesof freedom. The next challenge would be to perform suturing with twopairs of graspers manipulator and to perform more complicated NOTESprocedures like cholycystectomy and splencetomy with the MASTER system.

Example 3

In order to have a successful application of MASTER in NOTES, ESD wasdone. With the robotic manipulator, intensive experiments were conductedto verify the feasibility of the robotic system. Together with the helpof experienced endoscopists, 15 ex vivo ESDs, 5 in vivo ESDs and 2 invivo NOTES had been performed successfully on pigs. Before the trials,practice sessions with ex vivo pigs' stomachs were conducted with thesurgeon to establish the necessary steps for the ESD and NOTES. It alsoenabled the endoscopists to understand the capability and limitationfrom the endoscope and the robot.

Since the prototype is used only on animals, the robots are tentativelyjust cleaned thoroughly with soap, water and brush and subsequentlyreused for further trials. In future the robotic manipulator could bedesigned to be disposable after a single use to ensure it is sterilizedeffective for human patients.

ESD with MASTER

The finalized steps for robotic ESD are given as follows. FIG. 16provided is the actual view recorded from the endoscope during one ofthe ESD.

Firstly, the endoscopist has to spot the lesions where the ESD shouldtake place with a conventional endoscope. When the surgeon has locatedthe lesion, he proceeds with marking of the surroundings of the fleshwith a conventional needle knife set at coagulating mode. This is toensure both the endoscopist and surgeon are clear about where theprocedure is worked on and do not cut too excessively or too little.

Next the endoscopist uses an injector to inject saline at the lesion toseparate the muscle and mucosal layer. This procedure is to ensure thetool does not overcut into the muscle layer and cause excessive damageand bleeding to the patient. The saline is also colored with MethyleneBlue for better vision clarity during the subsequent procedures.

After the injection, the conventional endoscope is taken out andreplaced with the robotic manipulator. Both the endoscopist and surgeonthen try to perform the peripheral cut on the lesion. This cut isperformed using the robotic system to cut a complete circumferencearound the lesion. This makes the lesion region loose from thesurrounding and therefore easier to be manipulated by the robot. Theperipheral cut also ensures the cut is localized within the region andnot cut excessively into the other healthy site.

For this procedure, the endoscopist and surgeon try to position the hookslightly above the lesion before the hook pokes into the lesion usingelectro-cautery. Once the hook is through, the endoscopist then movesthe endoscope and the hook to cauterize along the surrounding of thelesion. During this time, the surgeon has to change the orientation ofthe hook if necessary to facilitate the peripheral cut.

After the peripheral cut, the endoscopist and surgeon has to go aroundthe lesion to ensure that the peripheral cut for the whole circumferenceis complete. If there is a site which is still attached to the lesion,the surgeon then try to finish the cut with the hook. This step isimportant since any remaining ridges on the lesion can cause thesubsequent steps to be more difficult.

After the peripheral cut is complete, the endoscopist proceeds with theactual removal of the lesion. The robotic manipulator then goes to thetop left hand corner of the lesion and the gripper grasps onto theformer. This exposes the flesh below the mucosal layer and the hook canproceed with cauterizing the lesion off. If there is a need, theendoscopist can relocate to another location for the robotic manipulatorto work on. The surgeon continues cauterizing until the whole lesion iscut loose from the stomach. If there is bleeding, the hook can acts as acoagulator to seal the blood vessels.

The view of the site after the procedure can be seen in the abovefigure. No perforation of the stomach is observed and the marked lesionis cleanly removed. ESD procedure had been successfully performed withthe robotic system.

Tables 6 and 7 below show the summary of the results for the fifteen exvivo animal trials and five in vivo animal trials performed by thesystem.

TABLE 6 Results for 15 ex vivo ESD animal trials Experiment DissectionTime Tissue Size (mm) Number (min) Length Width Lesion 1A 10 50 40Lesion 1B 41 40 30 Lesion 1C 48 32 28 Lesion 2A 48 27 20 Lesion 2B 22 2318 Lesion 2C 15 46 25 Lesion 3A 8 25 21 Lesion 3B 26 40 27 Lesion 3C 4145 28 Lesion 4A 7 50 26 Lesion 4B 24 37 33 Lesion 4C 9 30 26 Lesion 5A22 48 26 Lesion 5B 25 39 26 Lesion 5C 12 29 24

TABLE 7 Results for 5 in vivo ESD animal trials Dissection Time (min)Length (mm) Width Pig 1 (Conventional) 14 29.1 22.2 Pig 1 (Robotic) 4038.6 25.2 Pig 2 (Conventional) 26 50.2 33.0 Pig 2 (Robotic) 29 33 30 Pig3 (Conventional) 34 34.7 32.9 Pig 3 (Robotic) 19 42.8 25.7 Pig 4(Conventional) 15 21.8 17.2 Pig 4 (Robotic) 12 33.0 29.8 Pig 5(Conventional) 12 28.4 22.9 Pig 5 (Robotic) 3 28.8 22.9

From the results shown, it was observed that initially the manipulatortook much more time in performing ESD as compared with conventionalmethods. However, after more practices, refinement of the procedure andimproved communication between the endoscopist and surgeon, the timetaken for the procedure reduced to 3 minutes compared with 12 minutesfrom the conventional ESD. The average size for the lesion is about35.24 mm by 26.72 mm. The procedure also shows no complication,perforation and the sample lesion taken out is in one piece. This studywas performed on live pigs and the results show the method is feasibleand could be an improvement in performing ESD.

NOTES with MASTER

The robotic system was used to perform liver wedge resection on the livepigs. After the system gains access into the live pig's stomach, theendoscopist tries to establish the position and orientation of thestomach before using the robot to perform gastrotomy. Gastrotomyrequires the robot to cut a hole through the stomach wall to access theperitoneal cavity of the pig.

Once the system is inside the peritoneal cavity, the manipulator facesthe liver and proceeds with the liver wedge resection. The endoscopistdetermines the site where the cut should take place. The cauterizingshould begin close to the edge of the liver instead of the edge toensure there is a tension at the top end of the cut tissue. During thisprocedure the gripper has to grasp the edge of the liver to providetension for the cauterizing cut to be effective. The surgeon andendoscopist then try to cauterize the liver till only the top and bottomedges are left.

After they ensure the cut in the middle is complete, the two edges arethen cut. The surgeon can choose to cut the top edge or bottom edge tocomplete the liver resection. In the first trial, the endoscopist andsurgeon chose to remove the top edge first before cutting through thebottom edge.

With the gripper still holding the cut liver wedge, the hook thenproceeds with the coagulation of the liver surface to stop the bleeding.The surgeon then removes the robot from the pig and retrieves the liverwedge for analysis. The perforated cut on the stomach wall can then bemended using conventional methods such as haemoclips etc.

Table 8 shows the results for the two trials for NOTES. The time takento cut through the stomach, cut off the liver wedge and coagulating tookapproximately 8-9 minutes.

TABLE 8 Results on time taken for liver wedge resection and size Time(mins) Length (mm) Width (mm) Pig 4 (Robotic) 8.5 21 10 Pig 5 (Robotic)8.2 14  8

Example 4 Tension Study of Tendon and Sheath

The motion of the slave manipulator is completely controlled by thesurgeon and therefore there is no autonomy for the motion of the slavemanipulator. Hence is it imperative for the surgeon to obtain thecorrect and necessary information to make the best decision in carryingout the task. Due to the limited depth perception from the 2D image, thesurgeon cannot tell if the slave manipulator is pushing at the wrongplace excessively.

To compensate for the loss of depth perception, it is envisaged thatforce feedback could be used to rely useful information about theprocedure back to the surgeon. However, due to size constraints, it isimpractical to attach even miniaturized and sophisticated sensors at theend effectors for the slave manipulators. Furthermore, the sensors wouldhave to be sterilized prior the surgical intervention. Therefore, amethod of force prediction of the tendon sheath mechanism, which doesnot require installing any sensor at the slave manipulator, is proposed.The method utilizes the theoretical modelling of the characteristic ofthe tendon sheath actuation to predict the distal force and elongationduring the various phases of the actuation. The force prediction methodrequires a set of external sensors located at the actuator housing. Theoutput reading of the sensor could be used to predict the forceexperienced by the end effectors. The result of the force prediction isused as the input for the actuator at the master console in order toprovide the force sensation to the surgeon. Therefore by using thisforce prediction method, the surgeon is able feel the force that theslave manipulators are exerting on the surroundings. This ensures thesurgeon does not cause unnecessary trauma to the patient body and alsoensures that the robotic system does not break down due to excessivetension on the tendon. With this force feedback system in place, it isexpected that the surgeon could perform a NOTES procedure in a faster,safer and more consistent manner.

In the following description, the sheath is assumed to be bent with aconstant radius of curvature as seen in FIG. 20. In our model p is thefriction coefficient between the sheath and the tendon, N is the normalforce the sheath is exerting on the tendon in this unit length, T is thetension of the tendon, C is the compression force experienced by thesheath, T_(in) is the tension at one end of the sheath, R is the bendingradius of the tendon, x is the longitudinal coordinate from the housingend of the sheath to the present location, F is the friction between thetendon and the sheath in this unit length. To simplify our model, μ canbe assumed as the dynamic friction when the tendon is moving within thesheaths and it is a constant.

Using the force balance equations on the tendon for a small section dx,corresponding to the angle dα, we have

$\begin{matrix}{{{{T\mspace{14mu} d\; \alpha} = {- N}},{{d\; \alpha} = {{dx}/R}},{F = {{\mu \; N\mspace{14mu} {and}\mspace{14mu} {dT}} = F}}}{{thus}\mspace{14mu} {obtaining}}{\frac{T}{x} = {\frac{\mu}{R}T}}{T(x)} = {T_{in}^{{- \frac{\mu}{R}}x}}} & (1)\end{matrix}$

Next, the force balance equation is applied on the sheath. Forces N andF for the tendon are equal and opposite to the forces N and F of thesheath since they are reacting against each other. Since the tendonthickness is close to the inner diameter of the sheath and the segmentof tendon sheath is significantly small, the angle of both tendon andsheath is assumed to be the same. Therefore we have

C=−T,dC=−dT  (2)

The compressive force measured at the proximal end of the sheath is thesame as the tension measured from the tendon at the same end. Thisresult is easily verified by experiments.

The theory presented so far applies only to sheath and tendon with afixed curvature throughout its length, as shown in FIG. 21. In general,the sheaths are free to move and the curvature is different throughoutthe whole length. This is modeled as a sheath having n sections, eachhaving a different radius of curvature R₁ to R_(n) and a displacement ofx₁ to x_(n) from the housing. In this case equation (1) becomes

$\begin{matrix}{{{T(x)} = {T_{in}\left( ^{{{- \frac{\mu}{R_{1}}}x_{1}} - {\frac{\mu}{R_{2}}{({x_{2} - x_{1}})}} - {\frac{\mu}{R_{n - 1}}{({x_{n - 1} - x_{n - 2}})}} - {\frac{\mu}{R_{n}}{({x - x_{n - 1}})}}} \right)}}\left( {x_{n - 1} < x < x_{n}} \right)} & (3)\end{matrix}$

To predict the tension at the end of the sheath, expression (3) can besimplified as

T _(out) =T _(in) e ^(−K)  (4)

where

$K = {\mu \left( {\frac{x_{1}}{R_{1}} + \frac{x_{2} - x_{1}}{R_{2}} + \ldots + \frac{x_{n} - x_{n - 1}}{R_{n}}} \right)}$

represents the effective friction between the tendon and sheath. It isimportant to note that, if the sheath does not change its shape, K is aconstant. It is impossible to determine x_(j) and R_(j), but there is away to make use of this equation as described later.

Another relevant parameter is the elongation of the tendon and sheathsystem under a certain force. The study is initially applied to a sheathwith a fixed bending radius applying it to a generic sheath. Using e asthe tendon elongation and E as the combined stiffness of the tendon andsheath,

$\begin{matrix}{{e(x)} = \frac{T(x)}{E}} & (5)\end{matrix}$

where the tendon tension varies with x. To obtain the total elongation,equation (5) must be integrated over the length of the tendon sheathsystem, thus obtaining

$\begin{matrix}{e_{total} = {\frac{1}{E}{\int_{0}^{x}{T_{in}^{{- \frac{\mu}{R}}x}\ {x}}}}} & (6)\end{matrix}$

This is actually the area under the graph of the tension distribution Tdivided over the constant E. The analytical solution is

$\begin{matrix}{e_{total} = {\frac{T_{in}R}{E\; \mu}\left( {1 - ^{{- \frac{\mu}{R}}x}} \right)}} & (7) \\{{Another}\mspace{14mu} {expression}\mspace{14mu} {is}} & \; \\{e_{total} = {\frac{R}{E\; \mu}\left( {T_{in} - T_{out}} \right)}} & (8)\end{matrix}$

T_(out) is the tension experienced by the tendon at the end effector.This result is slightly different from that of the prior art for twomain reasons. First, pretension is not required. Second, it does notmake the assumption that the force is evenly distributed within thesheath. When the system is used, it starts with zero or low pretension.In this case, two actuators are used to control one degree of freedominstead of the traditional one actuator per degree of freedom. This alsosimplifies the modelling of the problem, since only one tendon undergoesa tension at any given time.

The above derivation is used when both the tendon and sheath have aconstant bending radius. If the sheath is modelled as having n sections,each having a different radius of curvature R₁ to R_(n) and each havinga displacement of x₁ to x_(n), then

$\begin{matrix}{{e_{o\; 1} = {\frac{R_{1}}{E\; \mu}\left( {T_{in} - T_{o\; 1}} \right)}}{e_{total} = {\frac{T_{in}R}{E\; \mu}\left( {1 - ^{{- \frac{\mu}{R}}x}} \right)}}\left( {x = x_{1}} \right)} & (9)\end{matrix}$

Where e_(o1) is the elongation at x=x₁ and T_(o1) is the tension atx=x₁. Similarly

$\begin{matrix}{\mspace{79mu} {e_{total} = {\frac{T_{in}}{E\; \mu}\left\lbrack {{R_{1}\left( {1 - ^{{- \frac{\mu}{R_{1}}}x_{1}}} \right)} + {R_{2}{^{{- \frac{\mu}{R_{1}}}x_{1}}\left( {1 - ^{{- \frac{\mu}{R_{1}}}{({x_{2} - x_{1}})}}} \right)}}} \right\rbrack}}} & \; \\{\mspace{79mu} \left( {x = x_{2}} \right)} & \; \\{\mspace{79mu} \ldots} & \; \\{e_{total} = {\frac{T_{in}}{E\; \mu}\left\lbrack {{R_{1}\left( {1 - ^{{- \frac{\mu}{R_{1}}}x_{1}}} \right)} + {\sum\limits_{i = 2}^{n}{{R_{i}}^{{- \mu}{\sum\limits_{j = 1}^{i}\; \frac{x_{j - 1} - x_{j - 2}}{R_{j - 1}}}}\left( {1 - ^{{- \mu}\frac{x_{i} - x_{i - 1}}{R_{i}}}} \right)}}} \right\rbrack}} & (10) \\{\mspace{79mu} \left( {x = x_{n}} \right)} & \; \\{\mspace{79mu} {e_{total} = {M_{e}T_{in}}}} & (11)\end{matrix}$

Where

$M_{e} = {\frac{1}{E_{\mu}}\left\lbrack {{R_{1}\left( {1 - ^{{- \frac{\mu}{R_{1}}}x_{1}}} \right)} + {\sum\limits_{i = 2}^{n}\; {R_{i}^{{- \mu}{\sum\limits_{j = 1}^{i}\; \frac{x_{j - 1} - x_{j - 2}}{R_{j - 1}}}}\left( {1 - ^{{- \mu}\frac{x_{i} - x_{i - 1}}{R_{i}}}} \right)}}} \right\rbrack}$

where M_(e) represents the effective elongation constant of the tendonsheath system. It is a constant if the shape of the tendon and sheathremains the same.

The relationship of K and M_(e) with T_(in) is expressed in FIG. 22. Thefull line curve represents the actual tension distribution for a genericsheath at T_(in). The dotted line represents the approximate solutioncoming from equation (4) as displacement x increases. The value of M_(e)is proportional to the area under the straight line curve and it is anindication of sheath deformation. In the experimental setup section, theapproach to find K and M_(e) is discussed in detail. It should be notedthat just the values for the force at the two ends are necessary forcontrol.

By approximating the original curvature to the one characterized by K,the relationship between M_(e) and K can be retrieved by evaluating thearea underneath T_(in)e^(−K) multiplied by 1/E,

$\begin{matrix}{{M_{e}T_{in}} = {\frac{T_{in}L}{EK}\left( {1 - ^{- K}} \right)}} & (12) \\{^{- K} = {{{- \frac{M_{e}E}{L}}K} + 1}} & (13)\end{matrix}$

It is seen that M_(e) and K are dependent on each other and the value ofM_(e) is enough to approximate K and vice versa. E is the Young'smodulus or stiffness of the tendon and therefore is the same regardlessof the shape. There is no readily available solution for equation (13)and numerical methods, such as Newton-Raphson or the Golden Sectionmethod have to be used.

However, this result is relevant only when the system undergoes apulling phase. In the case the tendon is just released after beingpulled, the system does not immediately go into the release phase. Itundergoes a transitional phase from pulling to releasing. FIG. 23 showsthe tension distribution within the sheath as the housing force isgradually reduced. The first effect of a decrease in T_(in) is areduction of the tension within the sheath, while the tension at the endeffector is not affected. Let X′ be the distance from the proximal endwhere the highest amount of tension within the sheath. As T_(in)decreases, X′ moves further from the proximal end and closer to the endeffector. Only when X′ reaches the end of the sheath, then T_(out)starts to decrease. This is the so called “backlash” of the tendonsheath system.

During the transitional phase, T_(out) remains constant until T_(in)reduces to T_(in5), as shown in FIG. 23. Therefore

T _(out) =T _(in0) e ^(−K)(14)

where T_(in0) is the highest value of the tension recorded at thehousing before it starts to loosen.

Differently from tension, the elongation varies during the transitionalperiod. The area under the tension distribution curve is proportional tothe elongation of the tendon and sheath and it undergoes relevantvariations. This change in the area is observed under the shaded graphof FIG. 23.

The elongation of the tendon during the transitional phase is derived intwo steps. First, the displacement X′ is calculated. The second step isto evaluate the elongation under the shaded areas A and B of the curve.To perform this step, we take advantage from the curvature approximatedby K.

Using L as the length of the whole sheath,

$\begin{matrix}{{T_{in}^{\frac{K}{L}X^{\prime}}} = {T_{{in}\; 0}^{{- \frac{K}{L}}X^{\prime}}}} & (15) \\{X^{\prime} = {\frac{L}{2K}\ln \frac{T_{{in}\; 0}}{T_{in}}}} & (16)\end{matrix}$

When X′=L, T_(in)=T_(in0)e^(−2K) is the force that the tension at thehousing side has to release before T_(out) reduces. This value can becalculated and used online in order to infer if the system is stillworking within the backlash region.

Now a way to calculate the elongation coming from area A of FIG. 23 ispresented. Using A be the area underneath the tension distribution fromx=0 to x=X′,

${{Area}\mspace{14mu} A} = {\int_{0}^{X^{\prime}}{^{- \frac{K}{L}}\ {x}}}$

$\begin{matrix}{{{Area}\mspace{14mu} A} = {\frac{L}{K}\left( {1 - ^{{- \frac{K}{L}}X^{\prime}}} \right)}} & (17)\end{matrix}$

Similar to the derivation of equation (6), the elongation under Area Ais given by the area of the curve multiplied by the highest tension inA, divided by E. In this case, the highest tension within A is

$T_{in}{^{\frac{K}{L}X^{\prime}}.}$

The resulting equation becomes

$\begin{matrix}{e_{A} \approx {\frac{T_{in}L\; ^{\frac{K}{L}X^{\prime}}}{EK}\left( {1 - ^{{- \frac{K}{L}}X^{\prime}}} \right)}} & (18)\end{matrix}$

The elongation caused by area B is proportional to the area under thecurve from x=X′ to x=L.

$\begin{matrix}{{{{Area}\mspace{14mu} B} = {\frac{L}{K}\left( {1 - ^{{- \frac{K}{L}}{({L - X^{\prime}})}}} \right)}}{e_{B} \approx {\frac{T_{in}L\; ^{\frac{K}{L}X^{\prime}}}{EK}\left( {1 - ^{- {K{({1 - \frac{X^{\prime}}{L}})}}}} \right)}}} & (19)\end{matrix}$

The combined elongation for the two areas become

$\begin{matrix}{e_{total} \approx {\frac{T_{in}L}{EK}\left( {{2^{\frac{K}{L}X^{\prime}}} - 1 - ^{({{- K} + \frac{2{KX}^{\prime}}{L}})}} \right)}} & (20)\end{matrix}$

By substituting

$X^{\prime} = {\frac{L}{2K}\ln \frac{T_{{in}\; 0}}{T_{in}}}$

into the equation above, we obtain

e _(total)≈(2√{square root over (T _(in0) T _(in))}−T _(in) −T _(in0) e^(−K))(21)

Release Phase

If T_(in) keeps decreasing after the transitional phase, the tendonsheath goes into the release phase. As T_(in) decreases, T_(out) startsto decrease as well. The method to derive the end effector force and theelongation is by changing the side that is “pulling” to be at the endeffector instead of the housing. Therefore,

T _(out) =T _(in0) e ^(K)  (22)

e _(out) =M _(e) T _(in) e ^(K)(23)

Release to Pull Phase

Similar to the transition from pulling to release, the tension at theend effector remains the same throughout the whole phase.

T _(out) =T _(in0) e ^(−K)(24)

Where T_(in0) is the lowest point of the tension recorded at the housingbefore it starts to tighten. The steps to find the deformation of thetendon and sheath are the same as the transition phase from pulling torelease. The final equation for the elongation is shown below.

$\begin{matrix}{e_{total} \approx {\frac{L}{EK}\left( {T_{in} - {2\sqrt{T_{in}T_{{in}\; 0}}} + {T_{{in}\; 0}^{K}}} \right)}} & (25)\end{matrix}$

With this equation found, the force at the end effector and elongationof the tendon and sheath are found for all the different phases when thesystem is used as shown in FIG. 24.

Experimental Setup

At the housing, or proximal end, two Faulhaber 2642W024CR DC servomotorswith a gear head of 30/1S 134:1 ratio are placed. The two tendons, thatare used to control one DOF, are fixed to the two actuators separately.These actuators are set to position control and each of them uses therotary optical encoder HEDS-5540 A14 attached to the actuator to measurethe angular rotation. The combined resolution of the encoder with thegear head is 67000 lines per revolution. At the end effector side, ordistal end, another actuator is used under torque control. Its mainpurpose is to simulate the load that the end effector can apply to theenvironment. It also has the same rotary encoder attached to theactuator to measure the amount of movement made by the system. In themiddle, two tendons and sheaths are used together with an overtube toprevent buckling. The forces at both ends of the system are measuredwith donut shaped load cells LW-1020 from Interface. Although theymeasure the compression experienced by the sheaths instead of thetension from the tendons, the result is similar to measuring the tensionforce at the same end, as shown in equation (2). The elastic modulus ofthe tendons must be known beforehand either from the supplier ormeasured with a simple stiffness test. The tendons used are Asahi 0.27mm 7×7 Teflon coated wire rope with, a length of approximately 2 m whilethe sheath is Acetone flat wire coil with outer diameter of 0.8 mm andinner diameter of 0.36 mm with a length of 2 m. The bending radius forthe sheath and tendon is about 30 cm.

The rationale of using two actuators for a single DOF is to ensure onlyone tendon is providing a tension at a given time, while theantagonistic tendon is left loose. This provides the highest possibleforce from the tendon since it does not have to work against the othertaut antagonistic tendon.

Necessary Steps for Prediction

After the end effector reaches the site where it is designated to work,the global shapes of the tendon and sheath are fixed. The experimentalprocedure then goes as follows. First, it is necessary to determine thevalues of K and M_(e) for this particular shape of the sheaths. To doso, initialization is required. The actuator at one end of the Degree ofFreedom pulls one end of the tendon until the robotic manipulatorreaches the end of its motion. This is the point where further pullingof the tendon at one side off the joint does not result in a change ofthe rotation angle, as shown in FIG. 25. Regarding the actuator at theother side of the degree of freedom, it must minimize the pulling forceto prevent interference with the initialization procedure. This ispossible since it is controlled by a separate actuator and it can readthe load cell on its side, thus deciding either to pull or release thetendon accordingly.

Even when the joint at the distal end reaches the limit of its motion,the actuator on the pulling side can still continue to pull in thetendon at the proximal end. This length that is pulled can only comedirectly from the deformation of the tendon within the sheath. Usingequation (11),

${M_{e} = \frac{e_{out}}{T_{in}}},$

e_(out) is obtained from the encoder reading while T_(in) is the forceobtained from the load cell of the pulling side of at the proximal end.The value of M_(e) can then be found from the process of initialization.Using equation (13), the value of K can be found using numerical methodsuch as Newton Raphson. With these two values, the elongation and forceat the distal end can be easily approximated since both elongation andforce at the distal end is a simple function of T_(in). This is repeatedfor the opposite direction of the DOF as well to capture the specificvalue of M_(E) and K. During initialization, the user has to ensure thatthe robotic manipulator does not touch any object or walls of theenvironment. After initialization, the user can start to use the MASTERas desired and the force at the end effector and elongation can becomputed using the obtained K and M_(e) value.

An assumption is made regarding the use of constants K and M_(e). Afterthe system has reached the site, the user no longer moves the systemaround and the shape of the sheaths can be assumed to be fixed. Onlythen can constants K and M_(e) be used without producing much error. Theuser could always change the system position anytime, but the roboticsystem requires a quick initialization after every change.

The system is then tested by either torque control of the actuator atthe distal end or allowing the distal end to push against a hard objector spring. The result when the robotic manipulator is pushing against ahard non-deformable object is shown in the following.

First of all, the setup is initialized to obtain the K and M_(e). Theexperiment is then performed with three profiles as seen in FIG. 26. Inthe first profile, the actuator pulls the tendon till it reaches alimited force of approximately 20N on the housing. In the second phase,the tendon is released at the housing to about 5N while in the thirdphase, the tendon is pulled again back to 20N. These last two phasestest the pull-to-release, release, release-to-pull and pulling phasesand check the approximation with the actual reading. In the figure, thecurves on top are the readings from the housing load cell while thedotted curves below are the readings from the end effector load cell.

The comparison of the end effector force with the predicted force isshown in FIG. 27. The results are broken up into three phases. Thegraphs on top are the plots of actual vs predicted end effector forcewhile the graphs below are the actual vs predicted elongation plots. Thecontinuous lines in full are the actual sensed values while the dottedlines are the predicted values. For this experiment, the maximum fullscale error is approximately 7% while for the elongation, it isapproximately 3%. The higher error in the force reading is due to thefact that the value of M_(e) is found and then used to deduce K. Overallthe average full scale error is less than 2%.

Although this method is capable of sensing both the elongation and endeffector force, it could be applied only in cases whereby there islittle or no change in the sheath shape after initialization. Thegreater the shape of the sheath changes, the worse the predictionbecomes. If a great change of the shape of the sheath is suspected, itis recommended to reinitialize the system again to obtain updated valuesfor M_(e) and K. If the application force on the sheath undergoesregular displacements, then this method should not be used. However, ifthe application is not critical and does not require high accuracy, thensmall changes in the shape of the sheaths during usage can be tolerated.

The force predicted at the end effector is already the combined forcethe joint experienced. Therefore, it can directly be scaled to thecontroller without the need of any further calculation or conversion.

Whilst there has been described in the foregoing description preferredembodiments of the present invention, it will be understood by thoseskilled in the technology concerned that many variations in details ofdesign, construction and/or operation may be made without departing fromthe present invention.

1. A system for flexible endoscopy, comprising: a robotic manipulatorcomprising: a flexible member couplable to an endoscope and comprising atendon sheath system including a tendon in a sheath, the tendondeformable within the sheath such that the tendon and the sheath canexert forces upon and react against each other, the flexible memberhaving a first end and a second end; an arm coupled to the first end ofthe flexible member such that the arm is movable by the flexible member;an end effector coupled to and movable by the arm; a first actuatorcoupled to the second end of the flexible member and configured fordriving the arm; a set of sensors configured for sensing flexible memberparameters at the second end of the flexible member; and a forceprediction unit coupled to the set of sensors, the force prediction unitconfigured for: determining constant parameters based upon the flexiblemember parameters sensed at the second end of the flexible member,including determining an effective elongation constant M_(e) of thetendon sheath system; and predicting as a function of the flexiblemember parameters sensed at the second end of the flexible member (a)tendon sheath system elongation including tendon elongation and sheathelongation at the first end of the flexible member and (b) forceexperienced by the end effector.
 2. The system of claim 1, wherein theforce prediction unit's prediction of the tendon sheath systemelongation does not require any sensor at a slave manipulator comprisingthe tendon, the arm, and the end effector.
 3. The system of claim 1,wherein the force prediction unit's determination of M_(e) comprises:pulling the tendon from an end of the tendon connectable to the firstactuator until a joint of the end effector at an end of the tendonattached to the arm reaches a limit of motion; further pulling thetendon to deform the tendon; and determining M_(e) from the equation${M_{e} = \frac{e_{out}}{T_{in}}},$ where e_(out) is obtained from anencoder corresponding to the first actuator and T_(in) is obtained froma load cell corresponding to the first actuator.
 3. The system of claim1, wherein the force prediction unit's determination of constantparameters at the second end of the tendon sheath system furthercomprises determining K of the tendon sheath system using a numericalmethod to solve the equation$^{- K} = {{{- \frac{M_{e}E}{L}}K} + 1.}$
 4. The system of claim 1,wherein when the tendon is being pulled at an end of the tendonconnectable to the first actuator, the force prediction unit isconfigured to predict tendon elongation at an end of the tendon attachedto the arm using the formula e_(total)≈(2√{square root over(T_(in0)T_(in))}−T_(in)−T_(in0)e^(−K)) and/or the tendon tension at theend of the tendon attached to the arm using the formulaT_(out)=T_(in)e^(−K).
 5. The system of claim 1, wherein when the tendonis not being pulled at an end of the tendon connectable to the firstactuator and is undergoing a transitional phase after having beenpulled, the force prediction unit is configured to predict tendonelongation at an end of the tendon attached to the arm using the formulae_(out)=M_(e)T_(in)e^(K) and/or tendon tension at the end of the tendonattached to the arm using the formula T_(out)=T_(in0)e^(K).
 6. Thesystem of claim 1, wherein when the tendon is not being pulled at an endof the tendon connectable to the first actuator and is undergoing atransitional phase after having been pulled, the force prediction unitis configured to predict tendon elongation at an end of the tendonattached to the arm using the formula e_(out)=M_(e)T_(in)e^(K) and/ortendon tension at the end of the tendon attached to the arm using theformula T_(out)=T_(in0)e^(K).
 7. The system of claim 1, furthercomprising a controller configured for controlling the roboticmanipulator, the controller coupled to the first actuator and the forceprediction unit and comprising a second actuator configured to generateforce feedback at the controller based upon the predicted forceexperienced by the end effector.
 8. The system of claim 7, wherein thecontroller comprises a hand-held member configured for effectingmovements of the robotic manipulator, and wherein the force feedbackcomprises a resisting force generated in response to a clinician'smovement of the hand-held member.
 9. A method for using a flexibleendoscopy system, the flexible endoscopy system comprising a roboticmanipulator having (a) a flexible member couplable to an endoscope andcomprising a tendon sheath system including a tendon in a sheath, thetendon deformable within the sheath such that the tendon and the sheathcan exert forces upon and react against each other, the flexible memberhaving a first end and a second end, (b) an arm coupled to the first endof the flexible member such that the arm is movable by the flexiblemember, (c) an end effector coupled to and movable by the arm, (d) afirst actuator coupled to the second end of the flexible member andconfigured for driving the arm, and (e) a set of sensors configured forsensing flexible member parameters at the second end of the flexiblemember, the method comprising: sensing flexible member parameters at thesecond end of the flexible member; determining constant parametersincluding an effective elongation constant M_(e) of the tendon sheathsystem based upon the flexible member parameters sensed at the secondend of the flexible member; and predicting as a function of the flexiblemember parameters sensed at the second end of the flexible member (a)tendon sheath system elongation including tendon elongation and sheathelongation at the first end of the flexible member and (b) forceexperienced by the end effector.
 10. The method of claim 9, whereinprediction of the tendon sheath system elongation does not require anysensor at a slave manipulator comprising the tendon, the arm, and theend effector.
 11. The method of claim 9, wherein determining M_(e)comprises: pulling the tendon from an end of the tendon connectable tothe first actuator until a joint of the end effector at an end of thetendon attached to the arm reaches a limit of motion; further pullingthe tendon to deform the tendon; and determining M_(e) from the equation${M_{e} = \frac{e_{out}}{T_{in}}},$ where e_(out) is obtained from anencoder corresponding to the first actuator and T_(in) is obtained froma load cell corresponding to the first actuator.
 12. The method of claim9, wherein determining constant parameters at the second end of thetendon sheath system further comprises determining K of the tendonsheath system using a numerical method to solve the equation$^{- K} = {{{- \frac{M_{e}E}{L}}K} + 1.}$
 14. The method of claim 9,further comprising predicting tendon elongation at an end of the tendonattached to the arm using the formula e_(total)≈(2√{square root over(T_(in0)T_(in))}−T_(in)−T_(in0)e^(−K)) and/or predicting tendon tensionat the end of the tendon attached to the arm using the formulaT_(out)=T_(in)e^(−K) when the tendon is being pulled at an end of thetendon connectable to the first actuator.
 15. The method of claim 9,further comprising predicting tendon elongation at an end of the tendonattached to the arm using the formula e_(out)=M_(e)T_(in)e^(K) and/ortendon tension at the end of the tendon attached to the arm using theformula T_(out)=T_(in0)e^(K) when the tendon is not being pulled at anend of the tendon connectable to the first actuator and is undergoing atransitional phase after having been pulled.
 16. The method of claim 9,wherein when the tendon is not being pulled at an end of the tendonconnectable to the first actuator and is undergoing a transitional phaseafter having been pulled, the force prediction unit is configured topredict tendon elongation at an end of the tendon attached to the armusing the formula e_(out)=M_(e)T_(in)e^(K) and/or tendon tension at theend of the tendon attached to the arm using the formulaT_(out)=T_(in0)e^(K).
 17. The method of claim 9, further comprisinggenerating force feedback at a controller coupled to the first actuatorbased upon the predicted force experienced by the end effector.
 18. Themethod of claim 17, wherein the force feedback comprises a resistingforce generated in response to a clinician's movement of a hand-heldmember of the controller.